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f Method Of Signal Processing For Multi-Leva 



The present invention relates to a method of signal processing for multi-level, 
e.g.greyscale, imaging, and to using such a method for operating a display or spatial 
light modulator in which the instantaneous intensity distribution afforded by the 
5 display or modulator is binary in nature but which is altered in a manner such that the 
time averaged distribution effectively has, or appears to have, multiple intensity 
levels, e.g. of intensity. For display purposes, this means that the alteration must be 
sufficiently fast for averaging to occur at the eye, preferably avoiding any flicker. 
This requirement may or may not apply for other purposes. 

10 The signal processing method of the invention can be used in conjunction with any 
spatial light modulator capable of producing a binary image, including those 
comprising an array of individually addressable cells or pixels, and those where the 
binary image is produced by scanning of a modulated light beam, for example. The 
term " binary spatial light modulator " used herein is intended to encompass all such 

15 devices, whether they are used for display or other purposes, for example information 
recordal, and variable components (for example lenses, filters and diffraction 
gratings) in optical systems. The term is intended to cover passive modulators where 
an existing light beam is affected by the modulator, and also those which act as light 
sources, for example arrays of light emitters, and electroluminescent devices. 

20 The term "image" as used herein is used to denote any spatially varied light 
distribution, normally, but not necessarily, of light intensity, and its production or 
resulting distribution will be referred to by the term "display". 

Furthermore, although the term "grey scale" is used herein as denoting a multi-level 
distribution, it should be made clear that the term is used in relation to any colour, 
25 including white. In addition, although the methods, arrays, backplanes, circuitry etc. 
of the invention and its embodiment are described in relation to a single colour 
(monochrome images), including white, it is envisaged that variable colour images or 
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displays etc. will be produced in maimers known per se, such as. by spatially 
subdividing a single array into different colour pixels, superimposing displays from, 
differently coloured monochrome arrays for example by projection, or temporal 
multiplexing, for example sequential projection of red green and blue images. 

5 Temporally varying binary modulation to achieve a multiple intensity effect is known, 
and can be effected by the use of multiple bit planes. In such a scheme, an array of 
digitised values, of amplitudes corresponding to the grey scale values allocated to the 
pixels of the array, is decomposed into a multiplicity, of bit planes. This multiple bit 
plane technique may be used with any binary spatial light modulator as defined above. 

10 It is possible to decompose a n-level digitised grey scale image into a plurality of 
image planes, or bit planes, e.g. binary planes (see below), of equal duration. 
However, in a preferred form, known as a weighted bit plane technique, the durations 
of the bit planes are weighted, each bit plane being representative of one level 
(exponent) of the digitisation. This reduces the number of bit planes which need to be 

1 5 stored to synthesise an image, and can reduce addressing requirements somewhat. 

Although in certain cases, it would be possible to use digital bases other than 2, this 
complicates matters insofar as each bit plane is not binary and thus is not so easily 
stored. Furthermore, each location of such a bit plane would then have more than one 
non-zero value, and the variation in non-zero values across the bit plane would need 
20 to be taken into account for the durations of operation of each pixel (possibly by 
further decomposing the non-binary bit plane to two or more binary bit planes). The 
discussion below will be limited to binary weighting, but the principles set out in such 
a context are believed to be sufficient to enable the skilled person to extrapolate to 
other exponential bases if required or desired. 

2 5 Where the digitisation is binary, so that each bit plane is an array of digital Is and Os, 
it is then only necessary to display each bit plane for a total period proportional to its 
binary weighting to provide a time averaged image equivalent to the digitised grey 
scale image. 
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Where possible, it is convenient to display each binary bit plane once for the total 
duration necessary to contribute to the grey scale image, but it is also possible to 
display one or more of the bit planes a plurality of times, not necessarily sequentially, 
provided that the total time spent in displaying each bit plane, relative to the total time 
5 spent in displaying all the bit planes, is proportional to its binary weighting. 

The different bit planes for a grey scale image can be stored as sequential binary 
strings in a computer, and will be read out one at a time in any desired order after 
which they can be discarded unless the image needs to be repeated. It is 
computationally easiest to read out the bit planes in the order in which they have been 
1 0 stored, since then the only address which needs to be stored is the starting address of 
the first stored bit planes, all bit planes then being read out one at a time simply by 
clocking out a predetermined number of data bits in sequence for each bit planes. 

It might be possible immediately to replace bit planes that have been read by the bit 
planes for a succeeding image, particularly where the bit planes are being produced in 
15 real time. However, under other circumstances this could be difficult, and the set of 
bit planes for a successive image will then normally be stored elsewhere. In certain 
cases it would be possible to provide storage for just two bit planes one of which is 
written while the other is being read, and vice versa. 

It would also be possible to control the reading and/or writing processes so as to 
20 convert the image standards as desired, for example from line sequential to interlaced. 

As or after each bit plane is read from memory, it is then written, e.g. using the single 
pass scheme described below, and viewed over a period corresponding to its 
weighting so that the eye synthesises the intended grey scale image. The single pass 
scheme is preferred insofar as it merely over-writes the preceding bit plane without 
25 the need for a second pass, the associated front electrode switching and blanking 
pulses. The avoidance of lost time between successive valid images enables 
continuous illumination and the easier provision of bit planes of an accurately 
weighted duration. 
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In such a scheme, each pixel is subjected to a series of voltage pulses according to the 
point in the grey scale it represents (as in the number representing the grey scale level, 
and usually but not necessarily in that order). There are more points in the grey scale 
than there are applications of voltages, due to the weighting employed, which is 
5 advantageous since it reduces the time spent actually driving the array. Each applied 
voltage may be of the same or opposed polarity compared to the preceding voltage, 
and the same number of voltage pulses, equal to the number of bit planes (ignoring - 
polarity), is applied to each pixel to synthesise the image. 

For example, in a 64 level grey scale with binary weighting, there will be 6 bit planes 
10 with relative durations of 2 n t where n ranges from 0 to 5, and each pixel can be 
represented by a corresponding 6 digit binary number. 

However, double pass schemes below could alternatively be adapted for use in 
multiple or weighted bit plane schemes. 

To achieve dc balance, it would be possible to produce each binary bit plane by any 
15 binary imaging method which itself produces dc balance - for example by sterling 
from a blank image, writing, viewing and erasing the binary image by selective 
energisation (+V) and driven blanking (-V) of selected pixels only. 

However, in most or all of such schemes, the actual duration of the binary image is 
not directly proportional to the time allocated thereto, for example because of 
2 0 mtervening blanking steps, etc., leading to a degree of distortion in the binary nature 
of the bit plane periods, and hence the perceived grey scale values. While this could 
be compensated for if desired, it represents an additional complication. 

Recently there has been developed a novel spatial light modulator in the form of a 
smectic liquid crystal layer disposed between an active semiconductor backplane and 
25 a common front electrode. It was developed in response to a requirement for a fast 
and, if possible, inexpensive, spatial light modulator comprising a relatively large 
number of pixels (320 x 240 up to 640 x 480) with potential application not only as a 
display device, but also for other forms of optical processing such as correlation and 
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holographic switching. Depending on the manner in which it is driven, and the value 
of the applied voltage, the modulator may be driven at a line rate of at least 10MHz 
and a frame rate of up to 15 to 20kHz, requiring a data input of around 1 to 1.5 Gpixel 
per second. Typically, while the pixel address time is around 100 nanoseconds, the 
5 pixel will actually take around 1 to 5 microseconds to switch between optical states; 
and while overall frame writing time is of the order of 24 microseconds, the frame to 
frame writing period is around 80 microseconds. 

This spatial light modulator can be driven according to single pass schemes, in which 
the front electrode is placed at a potential of V/2 relative to the backplane pixels, 
1 0 which are switched to zero volts or V volts. 

Alternatively it can be driven according to double pass schemes in which in one pass 
the front electrode is placed at zero volts and selected pixels are turned ON by 
switching pixel elements of the backplane array to V volts, and in the other pass the 
front electrode is placed at V volts and selected pixels are turned OFF by switching 
15 elements of the array to zero volts. For pixels which are not in the process of being 
switched the elements of the backplane follow the voltage of Hie front electrode. To 
maintain the same potential difference therebetween, the voltage at all backplane 
pixel elements of the array is simultaneously switched as the voltage on the front 
electrode is changed between zero and V volts. 

20 Our copending International Patent Applications (PCT/GB99/04285, ref: P20957WO, 
priority GB9827952.4; PCT/GB99/04286 and PCT/GB99/04276, refs: P20958WO 
and P20958WO1, both priority GB9827965.6; PCT/GB99/04282, ref: P20959WO, 
priority GB9827900.3; PCT/GB99/04279, ref: P20960WO, priority GB9827901.1; 
PCT/GB99/04274, ref: P20961WO, priority GB9827964.9; PCT/GB99/04275, ref: 

25 P20962WO, priority GB9827945.8; and PCT/GB99/04260, ref: P20963WO, priority 
GB 9827944.1) relate to other inventive aspects associated with this spatial light 
modulator, including the single and double pass schemes referred to in the preceding 
paragraph. 
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The aforesaid spatial light modulator is ideally suited to the use of fee bit plane 
technique mentioned above. However, the present invention is not limited to liquid 
crystal modulators, but can be applied to any spatial light modulator as referred to 
above. 

5 One problem which arises, particularly when operating liquid crystal display and 
modulators, is that of maintaining a dc balance at individual pixels. Our copending 
International Patent Application (PCT/GB99/04260, ref: P20963WO) filed together 
with this application is directed to a weighted bit plane technique as described above 
in which at least some of the bit planes are modified, and relates to a method of grey 
1 0 scale imaging using a weighted bit plane technique, in which an n-digit binary number 
represents the intended grey level of each pixel location in an array of binary pixels, 
wherein at least one said binary number has an unequal number of Is and Os, said 
method comprising the step of altering the number to a closely adjacent value such as 
to reduce the inequality of Is and Os. That method has particular but not exclusive 

15 relevance to the production of effective grey scale intensity distributions for display 
purposes, where the effective duration of the binary images (length and/or number of 
repeats) is such that temporal integration thereof, for example by a viewer, gives the 
grey scale image. It finds particular but not exclusive application to liquid crystal 
spatial light modulators, and enables dc balance to be obtained or at least more closely 

2 0 approximated at each pixel. 

The weighted bit plane method as operated therein requires that relaxation of the 
liquid crystal pixels is negligible over the duration of the longest bit plane, and this is 
not always possible. In such a case, the bit planes can be refreshed during the bit 
plane period(s), but at the expense of dc balance. 

25 Basically, a refresh step comprises repeating the application of the same voltage as 
was applied at the start of the bit plane so as to restore the switched state of the pixel. 
It may even be that the nth power binary weighted bit plane needs to be refreshed 
(2 n -l) times subsequent to the first writing so that a 2 n grey scale will involve 2 n frame 
writes of binary images when the refresh writing stages are included. 
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In a refresh scheme, bit planes are read out more than once, depending on the duration 
thereof. Thus it is not possible to discard the bit plane until it has undergone its final 
reading. Furthermore, if each bit plane is repeatedly read for the requisite number of 
times before proceeding to the next bit plane, it is necessary to store the starting 
address of the two bit planes. 

For example, taking a simple case of three bit planes A, B and C, of relative durations 
4t, 2t and t respectively, it would be possible to read these out in the order 
AAAABBC. However, this necessitates storing the start addresses of each of the bit 
planes, apart from plane C which is read only once, in order that the correct place for 
the refresh readout may be reached. 

European Patent Application 0762375 illustrates a method where bit planes are 
repeated a plurality of times according to their weighting, but the occurrences of the 
higher weighted bit planes are distributed throughout the display (frame) period. The 
bitplane sequence is not such that a lower weighted bit plane is always preceded by 
the next higher weighted bit plane, as will occur in the present invention to be 
explained below. 

In addition, and perhaps more importantly, there are cases where it is necessary to 
rewrite the entire grey scale image before proceeding to a new image, where display 
times are long or relaxation is fast for example. In such a case it is necessary not only 
to store the start address of the bit plane next to be used, but also the start address of 
the first bit plane of the entire sequence, until that image information is no longer 
required. 

An improved method of readout in such cases makes it possible to avoid the storage 
of a plurality of start addresses. At the high speeds involved in reading out the images 
when using the spatial light modulator of the preferred embodiment, this apparently 
rriinor step can be computationally significant and advantageous. 

The present invention provides a method of image signal processing wherein a signal 
defining a pixellated multi-level image is defined by a first plurality of binary strings 
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in a memory, the strings having associated therewith respective weightings and 
defining respective bit planes each corresponding to a digitised pixelwise intensity 
distribution, such that the weighted pixelwise intensity distribution over all said 
plurality of bit planes corresponds to said multi-level image, 
5 the method being characterised in that at least a second plurality of said binary 

Strings associated with the highest weightings are stored in sequential locations in said 
memory in decreasing order of weighting and the method including the step of 
making a succession of read cycles from the stored strings, each read cycle consisting 
of reading one or more of the stored strings in sequence as stored, commencing with 
10 the string for the highest weighting, the numbers of the strings read in the read cycles 
being varied so that at the end of the said succession of read cycles each string of the 
second plurality has been read out a number of times proportional to its associated 
weighting. 

Preferably the multi-level image is a multi-intensity image. 

1 5 The said succession of read cycles is repeated, for example where display times are 
long or relaxation is fast as noted above. 

In one form of the method according to the invention all of the first plurality of strings 
are stored in sequential locations in said memory in decreasing order of weighting. In 
this case the succession of read cycles can be such that each string of the first plurality 
20 is read out a number of times proportional to its weighting. 

The invention extends to a method of imaging in which the method of signal 
processing according to the invention is performed and each string of the first 
plurality is displayed as its bit plane each time it is read during the succession of read 
cycles for substantially the same period. 

25 In another form of the method of signal processing according to the invention there is 
at least one said string with lower weighting, in addition to said second plurality, 
which is read out once during said succession of read cycles. In this case the invention 
also extends to a method of imaging by performing such a method of signal 

8 



Printed 01-02-2001 



AMENDED SHEET 



f 8 



31 ,"pCT'GB99/04277 'ART34AMDT ■DESC'- 

processing wherein each string of the first plurality is displayed as its bit plane each 
time it is read during the succession of read cycles for substantially the same period, 
wherein when it is read said at least one said string with lower weighting is displayed 
as its bit plane for a duration which is less than said period and proportional to its 
5 weighting relative to the weighting of the lowest order string of the second plurality. 

In a preferred imaging method the bit planes are displayed on a pixellated liquid 
crystal display. A small ac potential difference may of signal processing applied to 
pixels of the liquid display in periods when bit planes are not being written. 

Thus the present invention requires that the second plurality of the bit planes, or all 
10 the bit planes, are stored as binary strings in sequential locations in a memory in 
decreasing order of weighting. By cyclically reading out from the stored bitplanes in 
sequence, starting with the highest weighted plane but varying the number of planes 
actually read out in each cycle, each bit plane can be read out a plurality of times • 
proportional to or equal to its weighting. Where there are lower weighted planes in 
15 addition to the second plurality, these will be read out once, for duration(s) less than 
the lowest order bit plane of the first plurality. This can be done at any time, 
including a period or periods within the reading out of the second plurality, but is 
preferably performed before or after the entire second plurality has been read. 

Thus in a method according to the present invention the triple bit plane image 
2 0 exemplified above will be read out with read passes ABC (once), AB (once), and A 
(twice), which when combined can give an overall order, for example, of 
ABCABAA, or ABCAAAB or ABAAABC as desired. Only the start address needs to 
be stored since each read pass commences at the same place, and continues to an 
address determined by counters. 

25 While some grey scale and refresh schemes automatically provide dc balance, a 
further option for schemes which do not do this is to allow dc imbalance to 
accumulate, for example while writing images and then allowing them to relax, 
calculating the imbalance (e.g. in an accompanying computer simulation), and then 
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applying local dc voltages to the pixels of a magnitude and duration such as to 
provide zero average dc. 

It should be understood that there have been references above to a liquid crystal cell 
incorporating an addressable array, the methods of the invention may be used in 
5 relation to any binary spatial light modulator. Where the imaging device is a liquid 
crystal device, prolongation of the binary images used to synthesise the grey scale 
image may be achieved in known manner by the application of a small ac field 
between successive binary images. 
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CLAIMS 

1. A method of image signal processing wherein a signal defining a pixellated 
multi-level image is defined by a first plurality of binary strings in a memory, the 
strings having associated therewith respective weightings and defining respective bit 

5 planes each corresponding to a digitised pixelwise intensity distribution, such that the 
weighted pixelwise intensity distribution over all said plurality of bit planes 
corresponds to said multi-level image, 

the method being characterised in that at least a second plurality of said binary 
strings associated with the highest weightings are stored in sequential locations in said 

10 memory in decreasing order of weighting and the method including the step of 
making a succession of read cycles from the stored strings, each read cycle consisting 
of reading one or more of the stored strings in sequence as stored, commencing with 
the string for the highest weighting, the numbers of the strings read in the read cycles 
being varied so that at the end of the said succession of read cycles each string of the 

1 5 second plurality has been read out a number of times proportional to its associated 
weighting. 

2. A method according to claim 1 wherein the multi-level image is a multi- 
intensity image. 

3. A method according to claim 1 or claim 2 wherein the said succession of read 
2 0 cycles is repeated. 

4. A method according to any preceding claim wherein all of the first plurality of 
strings are stored in sequential locations in said memory in decreasing order of 
weighting. 

5. A method according to claim 4 wherein said succession of read cycles is such 
25 that each string of the first plurality is read out a number of times proportional to its 
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6. A method according to any one of claims 1 to 4 wherein there is at least one 
said string with lower weighting, in addition to said second plurality, which is read 
out once during said succession of read cycles. 

7. A method of imaging comprising the steps of performing the method 
5 according to any one of claims 1 to 5 and displaying each string of the first plurality 

as its bit plane each time it is read during the succession of read cycles for 
substantially the same period. 

8. A method of imaging comprising the steps of perforating the method 
according to claim 6 and displaying each string of the first plurality as its bit plane 

10 each time it is read during the succession of read cycles for substantially the same 
period, wherein when it is read said at least one said string with lower weighting is 
displayed as its bit plane for a duration which is less than said period and proportional 
to its weighting relative to the weighting of the lowest order string of the second 
plurality. 

15 9. A method of imaging according to claim 7 or claim 8 wherein the bit planes 
are displayed on a pixellated liquid crystal display. 

10. A method according to claim 9 wherein a small ac potential difference is 
applied to pixels of the display in periods when bit planes are not being written. 
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